Prompt Generator for Stable Diffusion#

Note

Install ekorpkit package first.

Set logging level to Warning, if you don’t want to see verbose logging.

If you run this notebook in Colab, set Hardware accelerator to GPU.

!pip install ekorpkit[art] exit()

Preparing the environment#

%config InlineBackend.figure_format='retina'
%load_ext autotime

from ekorpkit import eKonf

eKonf.setLogger("WARNING")
eKonf.set_cuda(device=0)
print("version:", eKonf.__version__)

is_colab = eKonf.is_colab()
print("is colab?", is_colab)
if is_colab:
    eKonf.mount_google_drive()
project_dir = eKonf.set_workspace(workspace="/content/drive/MyDrive/workspace/", project="ekorpkit-book")
print("project_dir:", project_dir)

INFO:ekorpkit.base:Setting cuda device to ['A100-SXM4-40GB (id:0)']
INFO:ekorpkit.base:Setting EKORPKIT_WORKSPACE_ROOT to /content/drive/MyDrive/workspace/
INFO:ekorpkit.base:Setting EKORPKIT_PROJECT to ekorpkit-book
INFO:ekorpkit.base:Loaded .env from /workspace/projects/ekorpkit-book/config/.env

version: 0.1.40.post0.dev8
is colab? False
project_dir: /content/drive/MyDrive/workspace/projects/ekorpkit-book
time: 3.64 s (started: 2022-11-09 02:16:32 +00:00)

Load a Generator and Generate Prompts#

from ekorpkit.models.art.prompt import PromptGenerator
from ekorpkit.models.art.stable import StableDiffusion

pgen = PromptGenerator(verbose=False)
sd = StableDiffusion()

2022-11-09 02:16:37.366550: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
INFO:ekorpkit.base:Loaded .env from /workspace/projects/ekorpkit-book/config/.env

time: 7.26 s (started: 2022-11-09 02:16:36 +00:00)

text_prompts = pgen.generate_prompts(prompt="examining documents")
text_prompts

['examining documents, by peter mohrbacher, highly detailed, sharp focus, intricate, smooth, illustration, trending on artstation,',
 'examining documents by ilya kuvshinov, by greg rutkowski, highly detailed, trending on artstation',
 'examining documents in blender and blender. blender. hyperdetailed. V-ray. Dark masterpiece. Trending on artstation. 8k. Detailed. Detailed. Hyperrealistic.',
 'examining documents in a black hole, mysterious and sinister atmosphere, volumetric lighting, 4k peter dinklage as zuko, portrait, close up, concept art, intricate details, highly detailed by greg',
 'examining documents by mike mignola, beautiful shadowing, 3 d shadowing, reflective surfaces, illustrated completely, 8 k beautifully detailed pencil illustration, extremely hyper - detailed pencil illustration, intricate, epic composition, masterpiece, bold complimentary colors',
 'examining documents in space, concept art, illustration, dramatic lighting, soft details, painting oil on canvas, octane render, HDR, 4k, 8k, HD',
 'examining documents, digital art, trending on pinterest, trending on cgstudio, the most beautiful image ever created, dramatic, subtle details, illustration painting by greg rutkowski, trending on artstation',
 'examining documents, by ilya kuvshinov, by greg rutkowski, concept art, perfect symmetry, hyper detailed, 8 k, octane, unreal engine 5',
 'examining documents, intricate details, in the style of simon stalenhag carl spitzweg and dan mumford, zaha hadid, tuomas korpi',
 'examining documents in a vault of high quality scientific papers, 8k in unreal engine 5. keyshot and octane render, hyper realism, ultra detailed, digital art, trending in artstation, cinematic lighting, studio quality, smooth render,']

time: 14.4 s (started: 2022-11-09 02:16:43 +00:00)

Batch Run#

batch_name = "batch-run-test"
random_seed = False
batch_run_params = {
    "text_prompts": text_prompts[:4],
    "guidance_scale": [5, 10],
    "num_inference_steps": [50,100],
}
batch_run_pairs = [
#     ("guidance_scale", "num_inference_steps"),
    ['text_prompts']
]
batch_results = sd.batch_imagine(
    batch_name=batch_name, 
    batch_run_params=batch_run_params, 
    batch_run_pairs=batch_run_pairs,
    random_seed=random_seed,
    seed=123,
    text_prompts="Brave and beautiful new world in Jeju, artstation matte painting.",
    num_samples=3,
    max_display_image_width=800,
)
# print(batch_results)

Prompt: examining documents, by peter mohrbacher, highly detailed, sharp focus, intricate, smooth, illustration, trending on artstation,
../../../_images/prompt-generator_9_1.png 
Prompt: examining documents by ilya kuvshinov, by greg rutkowski, highly detailed, trending on artstation
../../../_images/prompt-generator_9_3.png 
Prompt: examining documents in blender and blender. blender. hyperdetailed. V-ray. Dark masterpiece. Trending on artstation. 8k. Detailed. Detailed. Hyperrealistic.
../../../_images/prompt-generator_9_5.png 
Prompt: examining documents in a black hole, mysterious and sinister atmosphere, volumetric lighting, 4k peter dinklage as zuko, portrait, close up, concept art, intricate details, highly detailed by greg
../../../_images/prompt-generator_9_7.png 
time: 2min 35s (started: 2022-11-09 02:16:58 +00:00)

# run_config_path = "/workspace/projects/ekorpkit-book/.MyDrive/workspace/projects/ekorpkit-book/disco-imagen/outputs/stable-diffusion/batch-run-test/configs/batch-run-test_guidance_scale_num_inference_steps(0)_run_configs.yaml"
# results = sd.batch_collage(run_config_path, max_display_image_width=800)

Train a Generator#

pgen.train(model_name="ekorpkit/stable-prompts-2")

pgen.load_model(model_name="ekorpkit/stable-prompts-2")